Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy

In recent years, endomicroscopy has become increasingly used for diagnostic purposes and interventional guidance. It can provide intraoperative aids for real-time tissue characterization and can help to perform visual investigations aimed for example to discover epithelial cancers. Due to physical constraints on the acquisition process, endomicroscopy images, still today have a low number of informative pixels which hampers their quality. Post-processing techniques, such as Super-Resolution (SR), are a potential solution to increase the quality of these images. SR techniques are often supervised, requiring aligned pairs of low-resolution (LR) and high-resolution (HR) images patches to train a model. However, in our domain, the lack of HR images hinders the collection of such pairs and makes supervised training unsuitable. For this reason, we propose an unsupervised SR framework based on an adversarial deep neural network with a physically-inspired cycle consistency, designed to impose some acquisition properties on the super-resolved images. Our framework can exploit HR images, regardless of the domain where they are coming from, to transfer the quality of the HR images to the initial LR images. This property can be particularly useful in all situations where pairs of LR/HR are not available during the training. Our quantitative analysis, validated using a database of 238 endomicroscopy video sequences from 143 patients, shows the ability of the pipeline to produce convincing super-resolved images. A Mean Opinion Score (MOS) study also confirms this quantitative image quality assessment.Comment: Accepted for publication on Medical Image Analysis journa

Autoimmune pancreatitis/IgG4-associated cholangitis and primary sclerosing cholangitis – Overlapping or separate diseases?

Autoimmune pancreatitis is a recently described fibroinflammatory disease which is characterised by raised serum levels of IgG4 (in >70% of cases), and an IgG4-positive lymphoplasmacytic tissue infiltrate. A favourable and rapid clinical response to oral steroid therapy is often seen. Biliary involvement is common, and the term IgG4-associated cholangitis has recently been coined. The cholangiographic appearances of IgG4-associated cholangitis and primary sclerosing cholangitis can be difficult to differentiate. Moreover, raised levels of serum IgG4 have been recently found in 9% of patients with primary sclerosing cholangitis (a much higher frequency than for other gastrointestinal diseases), and those with raised levels appear to progress more rapidly to liver failure. Here we review the similarities and differences between the biliary disease in autoimmune pancreatitis and primary sclerosing cholangitis, and address the issue of disease overlap. Improvements in understanding the relationship between these conditions might lead to an enhanced understanding of the aetiopathogenesis, and improved treatment of both conditions

Spatial scaling of species abundance distributions

Copyright © 2012 The Authors. Ecography © 2012 Nordic Society Oikos.Species abundance distributions are an essential tool in describing the biodiversity of ecological communities. We now know that their shape changes as a function of the size of area sampled. Here we analyze the scaling properties of species abundance distributions by using the moments of the logarithmically transformed number of individuals. We find that the moments as a function of area size are well fitted by power laws and we use this pattern to estimate the species abundance distribution for areas larger than those sampled. To reconstruct the species abundance distribution from its moments, we use discrete Tchebichef polynomials. We exemplify the method with data on tree and shrub species from a 50 ha plot of tropical rain forest on Barro Colorado Island, Panama. We test the method within the 50 ha plot, and then we extrapolate the species abundance distribution for areas up to 5 km2. Our results project that for areas above 50 ha the species abundance distributions have a bimodal shape with a local maximum occurring for the singleton classes and that this maximum increases with sampled area size

Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction

Purpose: Probe-based Confocal Laser Endomicroscopy (pCLE) is a recent imaging modality that allows performing in vivo optical biopsies. The design of pCLE hardware, and its reliance on an optical fibre bundle, fundamentally limits the image quality with a few tens of thousands fibres, each acting as the equivalent of a single-pixel detector, assembled into a single fibre bundle. Video-registration techniques can be used to estimate high-resolution (HR) images by exploiting the temporal information contained in a sequence of low-resolution (LR) images. However, the alignment of LR frames, required for the fusion, is computationally demanding and prone to artefacts. Methods: In this work, we propose a novel synthetic data generation approach to train exemplar-based Deep Neural Networks (DNNs). HR pCLE images with enhanced quality are recovered by the models trained on pairs of estimated HR images (generated by the video-registration algorithm) and realistic synthetic LR images. Performance of three different state-of-the-art DNNs techniques were analysed on a Smart Atlas database of 8806 images from 238 pCLE video sequences. The results were validated through an extensive Image Quality Assessment (IQA) that takes into account different quality scores, including a Mean Opinion Score (MOS). Results: Results indicate that the proposed solution produces an effective improvement in the quality of the obtained reconstructed image. Conclusion: The proposed training strategy and associated DNNs allows us to perform convincing super-resolution of pCLE images

Safety Study of Photodynamic Therapy Using Talaporfin Sodium in the Pancreas and Surrounding Tissues in the Syrian Golden Hamster

Aim. To assess the safety of photodynamic therapy (PDT) using talaporfin sodium on the pancreas and surrounding organs in normal hamsters. Methods. Fluorescence microscopy documented talaporfin levels in liver, duodenum, and pancreas up to 24 hours after photosensitisation. Lesion size in liver 3 days after PDT (50 J, 5 mg/kg, variable drug-light interval (DLI)) was documented to optimise the DLI. Using optimum DLI, pancreas and surrounding organs were treated with laser fibre touching the surface and animals were killed at 3 or 21 days. Results. Peak fluorescence was seen in duodenum and pancreas at 15 mins (second lower peak at 2 hours). Liver fluorescence was consistently high (peak 1 hour) until after 4 hours. Optimum DLI was seen at 15 minutes. The pancreas was relatively resistant to direct PDT injury (small lesions at high doses) but surrounding stomach, duodenum, and liver were more susceptible with evidence of adhesions and full thickness damage (localised peritonitis and duodenal perforation at highest doses). Conclusion. The safety profile is similar to PDT with longer acting photosensitisers. The pancreas appears safe to treat, but care is required to avoid high light doses to the intestinal tract, particularly the duodenum

Contrast Enhanced-Magnetic Resonance Imaging as a Surrogate to Map Verteporfin Delivery in Photodynamic Therapy

The use of in vivo contrast-enhanced magnetic resonance (MR) imaging as a surrogate for photosensitizer (verteporfin) dosimetry in photodynamic therapy of pancreas cancer is demonstrated by correlating MR contrast uptake to ex vivo fluorescence images on excised tissue. An orthotopic pancreatic xenograft mouse model was used for the study. A strong correlation ([i]r=0.57 ) was found for bulk intensity measurements of T1-weighted gadolinium enhancement and verteporfin fluorescence in the tumor region of interest. The use of contrast-enhanced MR imaging shows promise as a method for treatment planning and photosensitizer dosimetry in human photodynamic therapy (PDT) of pancreas cancer

Trial-based cost-effectiveness analysis comparing surgical and endoscopic drainage in patients with obstructive chronic pancreatitis

Objective: Published evidence indicates that surgical drainage of the pancreatic duct was more effective than endoscopic drainage for patients with chronic pancreatitis. This analysis assessed the cost-effectiveness of surgical versus endoscopic drainage in obstructive chronic pancreatitis. Design: This trial-based cost-utility analysis (ISRCTN04572410) was conducted from a UK National Health Service (NHS) perspective and during a 79-month time horizon. During the trial the details of the diagnostic and therapeutic procedures, and pancreatic insufficiency were collected. The resource use was varied in the sensitivity analysis based on a review of the literature. The health outcome was the Quality-Adjusted Life Year (QALY), generated using EQ-5D data collected during the trial. There were no pancreas-related deaths in the trial. All-cause mortality from the trial was incorporated into the QALY estimates in the sensitivity analysis. Setting: Hospital. Participants: Patients with obstructive chronic pancreatitis. Primary and secondary outcome measures: Costs, QALYs and cost-effectiveness. Results: The result of the base-case analysis was that surgical drainage dominated endoscopic drainage, being both more effective and less costly. The sensitivity analysis varied mortality and resource use and showed that the surgical option remained dominant in all scenarios. The probability of cost-effectiveness for surgical drainage was 100% for the base case and 82% in the assessed most conservative case scenario. Conclusions: In obstructive chronic pancreatitis, surgical drainage is highly cost-effective compared with endoscopic drainage from a UK NHS perspective

Mulsemedia: State of the art, perspectives, and challenges

Mulsemedia-multiple sensorial media-captures a wide variety of research efforts and applications. This article presents a historic perspective on mulsemedia work and reviews current developments in the area. These take place across the traditional multimedia spectrum-from virtual reality applications to computer games-as well as efforts in the arts, gastronomy, and therapy, to mention a few. We also describe standardization efforts, via the MPEG-V standard, and identify future developments and exciting challenges the community needs to overcome

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT

PURPOSE: The registration of Laparoscopic Ultrasound (LUS) to CT can enhance the safety of laparoscopic liver surgery by providing the surgeon with awareness on the relative positioning between critical vessels and a tumour. In an effort to provide a translatable solution for this poorly constrained problem, Content-based Image Retrieval (CBIR) based on vessel information has been suggested as a method for obtaining a global coarse registration without using tracking information. However, the performance of these frameworks is limited by the use of non-generalisable handcrafted vessel features. METHODS: We propose the use of a Deep Hashing (DH) network to directly convert vessel images from both LUS and CT into fixed size hash codes. During training, these codes are learnt from a patient-specific CT scan by supplying the network with triplets of vessel images which include both a registered and a mis-registered pair. Once hash codes have been learnt, they can be used to perform registration with CBIR methods. RESULTS: We test a CBIR pipeline on 11 sequences of untracked LUS distributed across 5 clinical cases. Compared to a handcrafted feature approach, our model improves the registration success rate significantly from 48% to 61%, considering a 20 mm error as the threshold for a successful coarse registration. CONCLUSIONS: We present the first DH framework for interventional multi-modal registration tasks. The presented approach is easily generalisable to other registration problems, does not require annotated data for training, and may promote the translation of these techniques